Obstacle information acquisition system technical field

ABSTRACT

There is provided a technique for acquiring information on an obstacle regardless of whether an operation of avoiding the obstacle has been performed. An obstacle information acquisition system includes an image obtaining part that obtains a shot image of a road on which a vehicle is traveling; an image recognizing part that detects an obstacle on a road and a lane from the image; and an obstacle information acquiring part that acquires obstacle information in which a location of the obstacle is associated with the image, based on a passable width of a lane where the obstacle is located in the image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/JP2021/026924 filed Jul. 19, 2021, claiming priority based onJapanese Patent Application 2020-177046 filed Oct. 22, 2020, the entirecontents of which are incorporated in their entirety.

TECHNICAL FIELD

The present disclosure relates to an obstacle information acquisitionsystem.

BACKGROUND ART

Conventionally, it is known that a vehicle detects a falling object(obstacle) on a road. Patent Literature 1 describes that informationindicating whether a vehicle has performed an operation of avoiding afalling object and location information of the falling object aretransmitted from the vehicle to a server, and when the proportion ofvehicles having performed an operation of avoiding the same fallingobject is greater than or equal to a predetermined value, the servernotifies the following vehicles that the falling object should beavoided.

CITATIONS LIST Patent Literature

-   Patent Literature 1: JP 2020-087309 A

SUMMARY OF THE DISCLOSURE Technical Problems

However, in the conventional technique, it cannot be determined whethera falling object should be avoided in the following cases. For example,there are a case in which a vehicle has passed by a falling object atvery close range from the falling object (passed by performing a slightsteering operation) and a case in which a vehicle is traveling in a laneadjacent to a lane where a falling object is present (an avoidanceoperation is not performed).

The present disclosure is made in view of the above-described problem,and provides a technique in which information on an obstacle can beacquired regardless of whether an operation of avoiding the obstacle hasbeen performed.

Solutions to Problems

To provide the above-described technique, an obstacle informationacquisition system includes an image obtaining part that obtains a shotimage of a road on which a vehicle is traveling; an image recognizingpart that detects an obstacle on a road and a lane from the image; andan obstacle information acquiring part that acquires obstacleinformation in which a location of the obstacle is associated with theimage, based on a passable width of a lane where the obstacle is locatedin the image.

Namely, the obstacle information acquisition system identifies apassable width of a lane where an obstacle is located from an image, andacquires obstacle information in which the location of the obstacle isassociated with the image, based on the passable width of the lane.Thus, according to the obstacle information acquisition system, obstacleinformation can be acquired regardless of whether an avoidance operationhas been actually performed.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of an obstacleinformation acquisition system.

FIG. 2A is a diagram showing an example of a shot image and FIG. 2B is adiagram showing a plane including a vehicle width axis and a vehiclelength axis.

FIGS. 3A to 3C are flowcharts of an obstacle information acquisitionprocess.

FIGS. 4A and 4B are flowcharts of an obstacle information acquisitionprocess according to a second embodiment.

FIG. 5 is a flowchart of an obstacle information acquisition processaccording to a third embodiment.

FIG. 6 is a diagram showing an example of a shot image.

DESCRIPTION OF EMBODIMENTS

Here, embodiments of the present disclosure will be described inaccordance with the following order:

-   -   (1) First embodiment;        -   (1-1) Configuration of an obstacle information acquisition            system;        -   (1-2) Obstacle information acquisition process;    -   (2) Second embodiment;    -   (3) Third embodiment; and    -   (4) Other embodiments.

(1) First Embodiment

(1-1) Configuration of an Obstacle Information Acquisition System:

FIG. 1 is a block diagram showing a configuration of an obstacleinformation acquisition system according to the present disclosure. Inthe present embodiment, the obstacle information acquisition systemincludes an in-vehicle system 100, a server 200, and an in-vehiclesystem 300. The in-vehicle system 100 is provided in a vehicle thatfunctions as a probe vehicle in the present embodiment, and has afunction of shooting an image of an area around the vehicle andtransmitting the image to the server 200. The in-vehicle system 300 isprovided in a vehicle (guidance vehicle) that provides guidance based oninformation delivered from the server 200.

The in-vehicle system 100 includes a control part 120, a recordingmedium 130, a camera 140, a GNSS receiving part 141, a vehicle speedsensor 142, a gyro sensor 143, a user I/F part 144, and a communicationpart 145. The control part 120 is a computer including a CPU, a RAM, aROM, etc. The in-vehicle system 100 can execute programs stored in therecording medium 130 and the ROM in the control part 120. The recordingmedium 130 has map information 130 a and vehicle body information 130 brecorded therein in advance. In addition, the recording medium 130records image information 130 c including an image shot in the course oftraveling and information associated with the image.

The map information 130 a is information used, for example, to identifya current location of the probe vehicle or provide route guidance, andincludes, for example, node data representing the location, etc., of anode set on a road on which the probe vehicle travels, shapeinterpolation point data representing the location, etc., of a shapeinterpolation point for identifying the shape of a road between nodes,link data representing a link between nodes, and ground object datarepresenting the location, shape, etc., of a ground object present on aroad or around the road. Note that in the present embodiment, the noderepresents an intersection. In addition, link data is associated withinformation indicating the number and types of lanes present on a roadsection represented by the link data, and the widths of the lanes. Inthe present embodiment, a location indicated by a node or a shapeinterpolation point indicates the location of a centerline on a roadsection, and by the location, the number of lanes, and the widths of thelanes, the locations of the lanes and an area in which the lanes arepresent can be identified. The vehicle body information 130 b isinformation on a vehicle body of the probe vehicle having the in-vehiclesystem 100 mounted thereon, and includes information indicating thedimensions of the vehicle body, such as the overall length, overallwidth, and height of the probe vehicle.

The GNSS receiving part 141 is a device that receives Global NavigationSatellite System signals, and receives radio waves from navigationsatellites and outputs, through an interface which is not shown, asignal for calculating a current location of the probe vehicle. Thecontrol part 120 obtains the signal, thereby obtaining a currentlocation (latitude, longitude, etc.) of the probe vehicle in a mapcoordinate system. The vehicle speed sensor 142 outputs a signalcorresponding to the rotational speed of wheels provided on the probevehicle. The control part 120 obtains the signal through an interfacewhich is not shown, thereby obtaining vehicle speed. The gyro sensor 143detects angular acceleration of the probe vehicle for a turn in ahorizontal plane, and outputs a signal corresponding to the orientationof the probe vehicle. The control part 120 obtains the signal, therebyobtaining a traveling direction of the probe vehicle. The vehicle speedsensor 142, the gyro sensor 143, and the like, are used to identify atravel path of the probe vehicle, and in the present embodiment, acurrent location is identified based on the point of departure andtravel path of the probe vehicle, and the current location of the probevehicle identified based on the point of departure and the travel pathis corrected based on an output signal from the GNSS receiving part 141.

The camera 140 is a device that obtains an image in a field of vieworiented toward the front of the probe vehicle. An optical axis of thecamera 140 is fixed on the probe vehicle, and the direction of theoptical axis is to be known by the in-vehicle system 100. In the presentembodiment, the camera 140 is mounted on the probe vehicle in a posturein which a vehicle width direction of the probe vehicle is perpendicularto optical axis center and an area ahead in a traveling direction of theprobe vehicle is included in the field of view. The control part 120obtains an image outputted from the camera 140 and analyzes the imageby, for example, extracting features, thereby being able to detect animage of a detection target included in the shot image.

The user I/F part 144 is an interface part for accepting, as input,instructions from a user and providing various types of information tothe user, and includes a touch panel type display, a speaker, etc.,which are not shown. Namely, the user I/F part 144 includes an outputpart for images and audio; and an input part for instructions from theuser. The output part of the user I/F part 144 can function as aguidance part that provides guidance on any information by an outputfrom the output part (a user I/F part 344 which will be described lateris also the same). The communication part 145 includes a circuit forperforming radio communication with other devices. In the presentembodiment, the control part 120 can give and receive various pieces ofinformation to/from the server 200 by radio communication through thecommunication part 145.

The control part 120 can execute a navigation program (not shown) thatdisplays a map including a current location or provides guidance on aroute to a destination. In addition, when the control part 120 detectsan obstacle present on a road from an image shot with the camera 140,the control part 120 can implement a function of acquiring obstacleinformation in which the location of the obstacle is associated with theimage. The function can be implemented by an obstacle informationacquisition program 121. To implement the function, the obstacleinformation acquisition program 121 includes an image obtaining part 121a, an image recognizing part 121 b, and an obstacle informationacquiring part 121 c. A guidance control part 121 d will be described ina third embodiment.

The image obtaining part 121 a is a program module that allows thecontrol part 120 to perform a function of obtaining a shot image of aroad on which the probe vehicle is traveling. In the present embodiment,the control part 120 controls the camera 140 every certain period in thecourse of probe vehicle's travel, to shoot a view including a road aheadof the probe vehicle. An image outputted from the camera 140 by theshooting is recorded as image information 130 c in the recording medium130. Note that in the present embodiment, the control part 120 obtains acurrent location of the probe vehicle based on output signals from theGNSS receiving part 141, the vehicle speed sensor 142, and the gyrosensor 143, and records, as image information 130 c, a current locationof the probe vehicle obtained at the time of shooting an image and ashooting time in the recording medium 130 such that the current locationand the shooting time are associated with each other. Note that thecontrol part 120 obtains, by a map matching process, a road section onwhich the probe vehicle is traveling. Note also that the control part120 can obtain a lane in which the probe vehicle is traveling, bydetecting the location of a section line such as a lane boundary line, acenterline, or a roadway outer line by an image recognition processwhich will be described later.

The image recognizing part 121 b is a program module that allows thecontrol part 120 to implement a function of detecting an obstacle on aroad and lanes from an image. The control part 120 detects a detectiontarget by performing an image recognition process. In the presentembodiment, the detection target includes a nearby vehicle (a passengercar, a truck, a bus, a motorcycle, etc.), a traffic sign, a trafficlight, a structure (a utility pole, a guardrail, etc.) around a road, apavement marking (characters, a crosswalk, a centerline, a lane boundaryline, a roadway outer line, etc.), etc.

The control part 120 obtains image information 130 c and performs lensdistortion correction, etc. The control part 120 recognizes a linepainted on a road, such as a roadway outer line, a lane boundary line,or a centerline, in an image having been subjected to the distortioncorrection. A section line recognition process may be performed byvarious techniques. For example, there is a process in which the controlpart 120 performs a straight-line detection process that uses Houghtransform, etc., and when the color of a region sandwiched betweendetected straight lines is a predetermined color such as white and thewidth of the region is within a preset distance, the control part 120recognizes the region as a line painted on a road, such as a laneboundary line, etc., or a centerline.

Furthermore, the control part 120 performs an image recognition processfor detecting the above-described detection target, using You Only LookOnce (YOLO), pattern matching, etc. As a result of the image recognitionprocess, the control part 120 detects an image of a detection targetfrom a shot image. Note that the above-described section linerecognition process may be performed using YOLO, pattern matching, etc.For example, when the control part 120 recognizes a nearby vehicle whichis one of detection targets, the control part 120 identifies, in animage, a bounding box that encloses the nearby vehicle. The size andlocation of the bounding box indicate the size of an image of the nearbyvehicle and the location of the nearby vehicle in the shot image. FIG.2A is a diagram showing an example of an image I shot with the camera140 and having been subjected to distortion correction. As shown in FIG.2A, in the present embodiment, a bounding box B is a rectangular regionthat encloses a nearby vehicle detected in the image I.

The size and location of the bounding box B are represented by, forexample, the coordinates of an upper left vertex and the coordinates ofa lower right vertex of the bounding box B. The control part 120 obtainsa height h (the number of pixels) of the bounding box B andrepresentative coordinates Bo (x, y) of the bounding box B from thecoordinates of the two diagonal vertices of the bounding box B. Therepresentative coordinates Bo are, for example, the center coordinatesof the bounding box B (the midpoint in a width direction and a heightdirection). The control part 120 identifies a relative orientation ofthe nearby vehicle as viewed from the probe vehicle, based on thelocation of the representative coordinates Bo of the bounding box B. Inaddition, the control part 120 identifies a distance from the probevehicle to the nearby vehicle, based on the height h of the bounding boxB and the type of the nearby vehicle.

Specifically, each set of coordinates in the image I is associated witha relative orientation of an object shown at the set of coordinates withrespect to the probe vehicle, and information indicating acorrespondence is stored in the recording medium 130. Based on thecorrespondence, the control part 120 obtains a relative orientation ofthe nearby vehicle shown at the representative coordinates Bo. In thepresent embodiment, a vehicle coordinate system with respect to theprobe vehicle is defined. The vehicle coordinate system is a coordinatesystem defined by a vehicle width axis (an X-axis shown in FIG. 2B) anda vehicle length axis (a Y-axis shown in FIG. 2B) which are orthogonalto each other.

FIG. 2B shows a plane including the vehicle width axis and the vehiclelength axis. In the drawing, the point O is the origin of the vehiclecoordinate system of the probe vehicle. In an example of FIG. 2B, thevehicle length axis is parallel to a link indicating a road section onwhich the probe vehicle travels. A relative orientation is representedby, for example, an angle (θ) formed by a straight line SL that connectsthe origin O of the vehicle coordinate system to a point P correspondingto the representative coordinates Bo, and the vehicle length axis (e.g.,when θ has a negative value, it indicates the left side of the vehiclelength axis when facing ahead in a traveling direction, and when θ has apositive value, it indicates the right side).

Furthermore, the control part 120 identifies the type of the nearbyvehicle in the bounding box B by an image recognition process. Thenearby vehicle may be categorized into types: e.g., a bus, a truck, apassenger car, and a motorcycle. In addition, in the present embodiment,for each type of nearby vehicle, a representative vehicle height (e.g.,1.5 [m] for a passenger car) is defined. Furthermore, a straight-linedistance between the probe vehicle and the nearby vehicle and a height hof the bounding box B obtained when the nearby vehicle is shot with thecamera 140 are measured in advance. For each type of nearby vehicle,there is stored, in the recording medium 130, information indicating acorrespondence between the height h of the bounding box B and thestraight-line distance with respect to the origin of the vehiclecoordinate system.

For example, when the height of a bounding box that encloses a passengercar whose representative actual measurement of vehicle height is 1.5 [m]is an h1 pixel, the h1 pixel is associated with the straight-linedistance “D1 [m]”, and when the height is an h2 pixel, the h2 pixel isassociated with the straight-line distance “D2 [m]”. For each of othertypes such as a bus, a truck, and a motorcycle, too, informationindicating a correspondence is stored in the recording medium 130. Basedon the correspondence, the control part 120 calculates a straight-linedistance D (see FIG. 2B) associated with the height h of the boundingbox B. In the above-described manner, based on an image shot with thecamera 140, the control part 120 obtains a relative orientation θ of anearby vehicle included in the image, and a straight-line distance Dbetween the nearby vehicle and the probe vehicle.

In the present embodiment, an image is shot every shooting period of thecamera 140, and for each image, a nearby vehicle is identified and astraight-line distance and a relative orientation are identified. Thus,the same nearby vehicle can be recognized in the course of shooting forseveral frames. Hence, in the present embodiment, while the same nearbyvehicle is shot, the control part 120 provides the same identificationinformation to the nearby vehicle. Hence, the control part 120identifies a feature (e.g., color or a pattern in a bounding box B) ofan image of each nearby vehicle whose relative orientation θ andstraight-line distance D have been identified, and records, in therecording medium 130, identification information (e.g., a number)corresponding to the feature such that the identification information isassociated with the relative orientation θ, the straight-line distanceD, information indicating the feature of the nearby vehicle, and theshooting time of the image.

Every time an image is shot, the same identification information isprovided to the same nearby vehicle, and thus, the control part 120determines whether features of images associated with nearby vehiclesthat are recognized in an immediately preceding image and the mostrecent image match each other, by referring to the recording medium 130.When the features match each other, the control part 120 providesidentification information provided to the nearby vehicle in theimmediately preceding image also to the nearby vehicle recognized in themost recent image. As a result, the same identification information isprovided to the nearby vehicles that are continuously shot with thecamera 140. Needless to say, even if images with the same feature areshot in contiguous frames, the control part 120 may perform processes,e.g., when a distance between images of two nearby vehicles is greaterthan or equal to a threshold value, the control part 120 does notconsider the vehicles to be identical. At any rate, during a periodduring which images provided with the same identification informationare detected, it can be considered that the same nearby vehicle iscontinuously detected.

Subsequently, the control part 120 obtains a relative orientation θ, astraight-line distance D, and identification information correspondingto a feature, of each nearby vehicle recognized by performing an imagerecognition process. Namely, since the image recognition process isperformed every time image information 130 c is obtained, the controlpart 120 chronologically obtains, from results of image recognitionprocesses performed on pieces of image information 130 c shot during apreset period before the present time, relative orientations θ,straight-line distances D, and pieces of identification informationcorresponding to features that are obtained during the preset period.

In addition, the control part 120 obtains a current location (a currentlocation obtained at the time of shooting image information 130 c withthe camera 140) and a traveling orientation (an orientation indicated bythe vehicle length axis) of the probe vehicle that are associated witheach piece of image information 130 c for which a relative orientation θand a straight-line distance D are calculated. Then, based on therelative orientation θ, the straight-line distance D, the currentlocation of the probe vehicle, and the traveling orientation of theprobe vehicle, the control part 120 obtains the location of a nearbyvehicle in the map coordinate system. Namely, for each of nearbyvehicles associated with the same identification information, a locationin the map coordinate system is obtained.

Furthermore, the control part 120 identifies a road section and a lanewhere a nearby vehicle is located. Namely, the control part 120identifies, based on node data and shape interpolation data for a roadsection on which the probe vehicle is traveling and a road sectionaround the road section and the location of a nearby vehicle, a roadsection where the nearby vehicle is present, and identifies, based onnode data and shape interpolation data for the road section, thelocation of a centerline on the road section. Furthermore, the controlpart 120 identifies the widths of lanes on the road section based onlane information of the road section, by referring to the mapinformation 130 a. Then, the control part 120 obtains a lane where thenearby vehicle is located, based on a distance between the centerlineand the nearby vehicle. Note that a lane where the nearby vehicle islocated may be identified based on the line types of section lines,which are detected from an image, at both edges in a width direction ofa lane region where the nearby vehicle is located, and lane informationobtained from the map information 130 a.

When the amount of change in the location of a nearby vehicle havinggiven identification information is less than or equal to a thresholdvalue in pieces of image information 130 c shot during the preset periodbefore the present time, the control part 120 considers the nearbyvehicle to be a stopped vehicle. In addition, in the present embodiment,a nearby vehicle being stopped due to a traffic jam is not considered tobe an obstacle. Specifically, for example, when both the probe vehicleand a nearby vehicle oriented in the same traveling direction as atraveling direction of the probe vehicle are being stopped, the controlpart 120 does not consider the nearby vehicle being stopped to be anobstacle. On the other hand, when, while a nearby vehicle is beingstopped at a specific location, the probe vehicle and other nearbyvehicles oriented in the same traveling direction as a travelingdirection of the probe vehicle are moving, the control part 120considers the nearby vehicle being stopped to be an obstacle. Inaddition, for example, when a traveling direction of a lane where thenearby vehicle is located is the same as a traveling direction of theprobe vehicle and it is determined that the probe vehicle has passed bythe nearby vehicle but the nearby vehicle is continuously stopped, too,the control part 120 considers the nearby vehicle to be an obstacle.

Next, detection of an obstacle other than a vehicle will be described.The control part 120 identifies a region sandwiched between sectionlines in an image, as a road surface (lane) region. When there is apavement marking other than section lines in the image, the control part120 recognizes the pavement marking. Then, when the control part 120 hasdetected an object that is at least partially included in the roadsurface (lane) region and that is other than a pavement marking, thecontrol part 120 obtains a relative location of the object with respectto the probe vehicle. For example, the control part 120 can obtain astraight-line distance and a relative orientation between the object andthe probe vehicle from the coordinates in the image of, for example, acentral portion of the object. The control part 120 identifies thelocation of the object in the map coordinate system, based on thecurrent location and traveling orientation of the probe vehicle obtainedat the time of image shooting, the straight-line distance, and therelative orientation. Locations of the same object are chronologicallyobtained from pieces of image information 130 c shot during the presetperiod before the present time. When the amount of change in thelocation of the object is less than or equal to a threshold value, thecontrol part 120 considers the object to be an obstacle.

The obstacle information acquiring part 121 c is a program module thatallows the control part 120 to implement a function of acquiring, basedon a passable width of a lane where an obstacle is located in an image,obstacle information in which the location of the obstacle is associatedwith the image. In the present embodiment, a passable width of a lanewhere an obstacle is located is calculated based on an image, and whenthe passable width is less than or equal to a preset value, obstacleinformation is transmitted to the server 200. The passable width is thewidth of a portion of the lane that is not occupied by the obstacle outof the full width of the lane. The control part 120 identifies, from thelocation of the obstacle in the lane where the obstacle is located, aportion of the lane that is not occupied by the obstacle, and calculatesthe width in actual space of the portion. Specifically, for example, thecontrol part 120 obtains the coordinates, in an image of an obstacle, ofan edge portion in a horizontal direction of a bottom portion of theimage, and calculates the coordinates of a point of intersection of astraight line extending in the horizontal direction from the edgeportion in the image and a section line representing a boundary of alane where the obstacle is located. The control part 120 obtains thelength (the number of pixels) of a line segment that connects the edgeportion to the point of intersection. The “w” in FIG. 2A indicates theline segment in an example of an image including an obstacle. In theimage I, for each y-coordinate, a correspondence between the number ofpixels in an x-direction and a distance in actual space is defined inadvance, and the correspondence is recorded in the recording medium 130.The control part 120 can obtain an actual distance (passable width)between the edge portion of the obstacle and the section linerepresenting the lane boundary from the y-coordinates of the edgeportion and the point of intersection, the length (the number of pixels)of the line segment that connects the edge portion to the point ofintersection, and the correspondence.

The control part 120 determines whether the passable width is less thanor equal to a preset value. The preset value is a value for determiningwhether there is a possibility of traveling outside a lane where anobstacle is located, to avoid the obstacle. For example, it is possibleto adopt, as the preset value, any of values ranging from the value oflane width itself to a value obtained by adding a preset margin to anassumable maximum vehicle width. When the passable width is less than orequal to the preset value, the control part 120 acquires obstacleinformation in which at least one image including a target obstacleamong pieces of image information 130 c is associated with the locationof the obstacle, and transmits the obstacle information to the server200. The location of the obstacle includes coordinates in the mapcoordinate system, a link ID indicating a road section where theobstacle is located, and identification information of a lane where theobstacle is located. Note that the obstacle information may also includethe shooting date and time of the image and the location of the probevehicle obtained at the time of shooting. When the passable width isgreater than the preset value, in the present embodiment, the controlpart 120 does not transmit obstacle information to the server 200. Thus,in this case, in the present embodiment, a guidance vehicle does notprovide guidance on the obstacle.

Note that in the present embodiment, when the type of an obstacle is avehicle and the type of the vehicle obtained by performing imagerecognition process is a bus, the control part 120 does not calculate apassable width of a lane and does not transmit obstacle information tothe server 200. Note that it may be configured such that when adetermination is made as to whether a location at which a bus detectedas an obstacle is stopped is a bus stop, and the location is a bus stop,obstacle information is not transmitted to the server 200. When the busis stopped at a bus stop, the control part 120 considers the stop to bea temporary stop for getting on and off, and in the present embodiment,obstacle information about the bus is not transmitted to the server 200.As a result, an unnecessarily large number of pieces of obstacleinformation can be prevented from being transmitted to the server 200.Note that the location of a bus stop can be obtained based on the mapinformation 130 a, and by calculating the location of the bus based onthe location and orientation (traveling orientation) of the probevehicle and a shot image, and checking the location of the bus againstthe location of the bus stop, it can be determined whether the bus isstopped at the bus stop. Note that by detecting a sign, etc., includedin an image and indicating a bus stop, it is also possible to determinethat the location of the bus corresponds to the bus stop.

In addition, in the present embodiment, when the height of an obstacleis less than or equal to a reference height, the control part 120 doesnot transmit obstacle information about the obstacle to the server 200.When the height of an obstacle is less than or equal to the referenceheight (here, it is assumed that the obstacle is a thin, flat objectsuch as a flattened cardboard box, a flattened plastic bag, or a crushedempty can), it is considered that the vehicle may be able to travelwithout avoiding the obstacle, and thus, obstacle information is nottransmitted. As a result, an unnecessarily large number of pieces ofobstacle information can be prevented from being transmitted to theserver 200. Note that a correspondence between the coordinates of abottom portion of an object in an image, the number of pixels of theobject in a height direction (y-direction) of the image from the bottomportion, and the actual height of the object is recorded in advance inthe recording medium 130, and the height of an obstacle can be obtainedbased on the coordinates of the obstacle in an image, the number ofpixels of the obstacle in a height direction (y-direction) of the image,and the correspondence.

As described above, according to the present embodiment, when there is apossibility that a probe vehicle may travel veering off its lane toavoid an obstacle based on an image shot with the camera 140 included inthe probe vehicle, obstacle information in which the location of theobstacle is associated with the image can be transmitted to the server200. Hence, regardless of whether an avoidance operation has beenactually performed, obstacle information can be acquired. For example,obstacle information can also be acquired from an image shot by a probevehicle having passed through an obstacle by performing, for example,such a gentle steering operation that is not determined to be anavoidance operation, without performing an avoidance operation such assudden steering, or by a probe vehicle that has not performed anavoidance operation because the probe vehicle travels in a lane otherthan a lane where the obstacle is located (e.g., a next lane or anoncoming lane).

Next, a configuration of the server 200 will be described. In thepresent embodiment, the server 200 has a function of delivering obstacleinformation to a guidance vehicle. There may be a single server 200 orthere may be a plurality of servers 200. The server 200 includes acontrol part 220, a recording medium 230, and a communication part 240.The control part 220 is a computer including a CPU, a RAM, a ROM, etc.The control part 220 can execute an obstacle information acquisitionprogram 221 stored in the recording medium 230 or the ROM.

The communication part 240 includes a circuit that performscommunication with other devices. The control part 220 can perform radiocommunication with the in-vehicle systems 100 and 300, etc., through thecommunication part 240 by performing processes of the obstacleinformation acquisition program 221. The recording medium 230 has mapinformation 230 a recorded therein. The configuration of the mapinformation 230 a is common to that of the map information 130 a.

In the present embodiment, when the obstacle information acquisitionprogram 221 is executed, the control part 220 functions as an obstacleinformation acquiring part 221 a and a guidance control part 221 b. Inthe present embodiment, when the control part 220 receives, by afunction of the obstacle information acquiring part 221 a, obstacleinformation from the in-vehicle system 100 mounted on a vehicle thatfunctions as a probe vehicle, the control part 220 saves the obstacleinformation in the recording medium 230 (obstacle information 230 b). Inaddition, the control part 220 performs the above-described imagerecognition process on an image included in the obstacle information,thereby detecting an obstacle and calculating a passable width of a lanewhere the obstacle is located. When the control part 220 acquires piecesof obstacle information about an obstacle located at the same point froma plurality of probe vehicles, the control part 220 may, for example,calculate a passable width from each of images included in therespective pieces of obstacle information and calculate a statisticalvalue of the passable width. In addition, since there is a possibilitythat the location of the obstacle may have changed, a passable widthcalculated at or earlier than a certain time ago from the current timemay be excluded from the calculation of a statistical value.

The guidance control part 221 b is a program module that allows thecontrol part 220 to implement a function of providing guidance on anobstacle based on obstacle information. Note that in the presentembodiment, a guidance control part is implemented by cooperation of theguidance control part 221 b of the server 200 and a guidance controlpart 32 a of a guidance vehicle. In the present embodiment, the guidancecontrol part 221 b of the server 200 has a function of identifying avehicle located in a preset area including the location of an obstacle,delivering obstacle information to the vehicle, and allowing the vehicleto provide guidance based on the obstacle information. In the presentembodiment, the preset area including the location of an obstacle may beassumed to be an area having a preset distance in all directions withthe location of the obstacle being at the center thereof, an area whoseradius is a preset distance, a mesh including the location of theobstacle, etc.

To implement this function, the control part 220 obtains vehiclelocation information transmitted from an in-vehicle system that canperform communication (an in-vehicle system mounted on a vehicle thatcan serve as a guidance vehicle), and saves the location of the vehiclethat can serve as a guidance vehicle in the recording medium 230. Thecontrol part 220 sets a preset area including the location of anobstacle, based on obstacle information 230 b, and identifies vehicleslocated in the preset area. Note that the identified vehicles mayinclude a probe vehicle. Then, obstacle information is delivered to theidentified vehicles. In the present embodiment, the obstacle informationdelivered from the server 200 includes the location of the obstacle, aroad section and a lane where the obstacle is located, and a passablewidth (or a statistical value thereof). In the present embodiment, theobstacle information delivered from the server 200 may not include animage including the obstacle.

Next, a configuration of the in-vehicle system 300 mounted on a guidancevehicle will be described. The in-vehicle system 300 includes a controlpart 320, a recording medium 330, a camera 340, a GNSS receiving part341, a vehicle speed sensor 342, a gyro sensor 343, a user I/F part 344,and a communication part 345. These components included in thein-vehicle system 300 have the same functions as the components 140 to145 included in the in-vehicle system 100. In addition, theconfiguration of map information 330 a recorded in the recording medium330 is common to that of the map information 130 a. Vehicle bodyinformation 330 b is information on a vehicle body of the guidancevehicle having the in-vehicle system 300 mounted thereon, and includesinformation such as the overall length, overall width, and height of theguidance vehicle.

The control part 320 can execute a navigation program (not shown) thatdisplays a map including a current location of the guidance vehicle orprovides guidance on a route to a destination. In the presentembodiment, the control part 320 can execute an obstacle informationacquisition program 321 which is one of the functions of the navigationprogram. The obstacle information acquisition program 321 includes aguidance control part 321 a. The guidance control part 321 a is aprogram module that allows the control part 320 to implement a functionof providing guidance based on obstacle information. The control part320 obtains a current location of the guidance vehicle by a function ofthe guidance control part 321 a. In addition, it is configured such thatvehicle location information indicating a current location of theguidance vehicle is transmitted to the server 200 at preset timing.Furthermore, when the control part 320 acquires obstacle informationfrom the server 200, the control part 320 identifies, from the obstacleinformation, the location of an obstacle and a road section and a lanewhere the obstacle is located. When the guidance vehicle is traveling onthe road section and in the lane where the obstacle is located and hasnot yet passed through the location of the obstacle, the control part320 determines whether a passing width of the guidance vehicle isgreater than a passable width. Namely, the control part 320 obtains theoverall width of the guidance vehicle by referring to the vehicle bodyinformation 330 b, and calculates a passing width obtained by adding awidth for a preset margin to the overall width.

When the passing width of the guidance vehicle is greater than thepassable width which is received as the obstacle information, it isexpected that the guidance vehicle veer off to a next lane to avoid theobstacle, and thus, the control part 320 provides lane change guidanceto use, as a recommended lane, a lane other than a lane where theobstacle is located. When the passing width of the guidance vehicle isless than or equal to the passable width, it is determined that theguidance vehicle can avoid the obstacle without veering off to the nextlane, and the control part 320 provides guidance for calling attentionby providing notification of the presence of the obstacle. The passingwidth of a guidance vehicle varies depending on the overall width, etc.,of the guidance vehicle, and thus, as in the present embodiment, bycalculating, for each guidance vehicle, a passing width of the guidancevehicle and comparing the passing width with a passable width, lanechange guidance can be provided to a guidance vehicle that requires alane change.

The control part 320 displays an icon representing an obstacle at thelocation of the obstacle on a map displayed on a touch panel display ofthe user I/F part 344, thereby providing notification of the location ofthe obstacle. In addition, by calculating a distance K [m] along a roadfrom the guidance vehicle to the obstacle, the presence of the obstacleK [m] ahead may be notified by voice. In addition, when lane changeguidance is provided, the control part 320 provides notification of alane where the obstacle is located and a recommended lane different thanthe lane among lanes on a road section on which the guidance vehicle istraveling, by displaying a lane list diagram, etc.

Note that the in-vehicle system 300 of the guidance vehicle may alsohave an equivalent function to that of the in-vehicle system 100 of theprobe vehicle. Namely, it may be configured to transmit to the server200 obstacle information in which an image shot by the guidance vehicleupon passing through the location of an obstacle indicated by obstacleinformation transmitted from the server 200 is associated with thelocation of the obstacle. By doing so, the server 200 can recognize,based on the image transmitted from the guidance vehicle, that theobstacle is still present on the road (the obstacle has not been removedfrom the road). In addition, since the number of accumulated shot imagesof the same object increases, the reliability of the statistical valueof passable width calculated based on the images increases.

Note that it may be configured such that even if the control part 320 ofthe guidance vehicle analyzes an image that is shot with the camera 340at a location a preset distance behind the location of the obstacle(such a distance that allows the location of the obstacle to be includedin a field of view of the camera 340) and does not recognize thepresence of the obstacle in the image, the image is transmitted to theserver 200 such that the image is associated with the location of theguidance vehicle obtained at the time of shooting and the shooting dateand time. By doing so, when the server 200 obtains the image, theshooting date and time, and the shooting location from the guidancevehicle, the server 200 performs an image recognition process on theimage, and if the obstacle has been removed, then the server 200 canrecognize that the obstacle is not included in the image. When it isrecognized, based on one or more images, that an obstacle is notincluded in a shot image of a location where the obstacle has beenpresent before, the control part 220 of the server 200 may determinethat the obstacle has been removed from the road and terminate deliveryof obstacle information about the obstacle thereafter.

(1-2) Obstacle Information Acquisition Process:

Next, an obstacle information acquisition process performed by thecontrol part 120 in the in-vehicle system 100 of a probe vehicle will bedescribed with reference to FIG. 3A. The process of FIG. 3A is performedevery lapse of a certain period of time. When the obstacle informationacquisition process of FIG. 3A starts, the control part 120 obtains acurrent location of the probe vehicle (step S100). Namely, the controlpart 120 obtains a current location of the probe vehicle and a roadsection (link) on which the probe vehicle is traveling, by a mapmatching process that uses outputs from the GNSS receiving part 141, thevehicle speed sensor 142, and the gyro sensor 143 and the mapinformation 130 a. In addition, section lines included in an image shotwith the camera 140 are detected to identify a lane in which the probevehicle is traveling.

Subsequently, the control part 120 obtains a shot image by a processperformed by the image obtaining part 121 a (step S102). Namely, thecontrol part 120 obtains an image shot with the camera 140.Subsequently, the control part 120 performs image recognition by afunction of the image recognizing part 121 b (step S105). Namely, thecontrol part 120 performs an image recognition process for detecting adetection target, on the image obtained at step S102. Then, as describedabove, lanes and an obstacle are detected.

Subsequently, the control part 120 determines whether there is anobstacle, by a function of the image recognizing part 121 b (step S110).Namely, the control part 120 determines whether an obstacle has beendetected as a result of the image recognition process at step S105. Ifit is not determined at step S110 that there is an obstacle, then thecontrol part 120 ends the obstacle information acquisition process. Ifit is determined at step S110 that there is an obstacle, then thecontrol part 120 determines whether the obstacle is a vehicle, by afunction of the image recognizing part 121 b (step S115). Namely, thecontrol part 120 determines whether a vehicle that is continuouslystopped despite vehicles therearound traveling has been detected in theimage.

If it is determined at step S115 that the obstacle is a vehicle, thenthe control part 120 determines whether the obstacle is a bus (stepS120). Namely, the control part 120 determines whether the type of anearby vehicle obtained by performing the image recognition process is abus. If it is determined at step S120 that the obstacle is a bus, thenthe control part 120 ends the obstacle information acquisition process.

If it is not determined at step S115 that the obstacle is a vehicle,then the control part 120 determines whether the height of the obstacleother than a vehicle is greater than or equal to a reference height(step S125). Namely, the control part 120 calculates the height of theobstacle based on the number of pixels of the obstacle in a heightdirection of the image, the coordinates in the image of the obstacle,etc. If it is not determined at step S125 that the height is greaterthan or equal to the reference height, then the control part 120 endsthe obstacle information acquisition process.

If it is not determined at step S120 that the obstacle is a bus, i.e.,if the obstacle is a stopped vehicle other than a bus, e.g., a passengercar or a truck, then the control part 120 calculates a passable width bya function of the obstacle information acquiring part 121 c (step S130).In addition, when it is determined at step S125 that the height isgreater than or equal to the reference height, too, the control part 120performs step S130. At step S130, the width of a portion of the lanethat is not occupied by the obstacle out of the full width of the laneis calculated as a passable width.

After performing step S130, the control part 120 determines, by afunction of the obstacle information acquiring part 121 c, whether thepassable width is less than or equal to a preset value (step S135). Ifthe passable width is less than or equal to the preset value, then thecontrol part 120 transmits to the server 200 obstacle information inwhich the location of the obstacle is associated with the image (stepS140). In the present embodiment, the obstacle information includes atleast one image including the obstacle; the location of the obstacle(coordinates in the map coordinate system, a link ID indicating a roadsection where the obstacle is located, and identification information ofa lane where the obstacle is located); the shooting date and time of theimage; the location of the probe vehicle at the time of shooting; etc.If it is not determined at step S135 that the passable width is lessthan or equal to the preset value, then the control part 120 ends theobstacle information acquisition process.

FIG. 3B is a flowchart of an obstacle information acquisition processperformed by the server 200. The process of FIG. 3B is repeatedlyperformed every lapse of a certain period of time. When the process ofFIG. 3B starts, the control part 220 acquires obstacle information by afunction of the obstacle information acquiring part 221 a (step S200).Namely, the control part 220 acquires obstacle information received fromthe in-vehicle system 100 of a probe vehicle and obtains a road section(link) where an obstacle is located, from the obstacle information.

Subsequently, the control part 220 obtains vehicle location informationby a function of the guidance control part 221 b (step S205). Namely,the control part 220 obtains pieces of vehicle location informationtransmitted from the in-vehicle systems 300 of guidance vehicles.

Subsequently, the control part 220 transmits obstacle information to aguidance vehicle in a preset area including the location of the obstacle(step S210). Namely, the control part 220 identifies a guidance vehiclelocated in the preset area including the location of the obstacle, basedon the obstacle information acquired at step S200 and the vehiclelocation information obtained at step S205, and delivers obstacleinformation to the in-vehicle system 300 of the guidance vehicle.

FIG. 3C is a flowchart of an obstacle information acquisition processperformed by the in-vehicle system 300. The process shown in FIG. 3C isrepeatedly performed every lapse of a certain period of time. When theobstacle information acquisition process shown in FIG. 3C starts, thecontrol part 320 obtains a current location by a function of theguidance control part 321 a (step S300). Namely, the control part 320obtains a current location of the guidance vehicle and a road section(link) on which the guidance vehicle is traveling, by a map matchingprocess that uses outputs from the GNSS receiving part 341, the vehiclespeed sensor 342, and the gyro sensor 343 and the map information 330 a.In addition, the control part 320 detects section lines included in animage shot with the camera 340 to identify a lane in which the guidancevehicle is traveling.

Subsequently, the control part 320 acquires obstacle information by afunction of the guidance control part 321 a (step S305). Namely, thecontrol part 320 acquires obstacle information delivered from the server200. The obstacle information includes a road section and a lane wherean obstacle is located.

Subsequently, the control part 320 determines, by a function of theguidance control part 321 a, whether the guidance vehicle is travelingon the road section and in the lane where the obstacle is located andhas not yet passed through the obstacle (step S310). Namely, the controlpart 320 determines whether the guidance vehicle is traveling on thesame road section as the obstacle and traveling in the same lane as alane where the obstacle is located. When both the road section and thelane are the same as those of the obstacle, the control part 320determines whether the guidance vehicle has not yet passed through theobstacle or has already passed through the obstacle, based on thecurrent location of the guidance vehicle, the location of the obstacle,the shape of the road section, and a traveling direction of the lane.

If it is not determined at step S310 that the guidance vehicle istraveling on the road section and in the lane where the obstacle islocated and has not yet passed through the obstacle, then the controlpart 320 ends the process of FIG. 3C. If it is determined at step S310that the guidance vehicle is traveling on the road section and in thelane where the obstacle is located and has not yet passed through theobstacle, then the control part 320 determines, by a function of theguidance control part 321 a, whether a passing width of the guidancevehicle is greater than a passable width (step S315). Namely, thecontrol part 320 obtains a passing width of the guidance vehicle basedon the vehicle body information 330 b, and compares the passing widthwith a passable width obtained from the server 200. If it is determinedat step S315 that the passing width of the guidance vehicle is greaterthan the passable width, then the control part 320 provides lane changeguidance by a function of the guidance control part 321 a (step S320).If it is not determined at step S315 that the passing width of theguidance vehicle is greater than the passable width, then the controlpart 320 provides attention-calling guidance by a function of theguidance control part 321 a (step S325). Note that the guidance at stepS320 and S325 may be provided regardless of whether guidance on a routeto a destination is being provided.

(2) Second Embodiment

A second embodiment differs from the first embodiment in content ofobstacle information to be transmitted to the server 200 based on apassable width, which will be specifically described with reference toFIG. 3A. After calculating a passable width based on an image at stepS130, it is determined whether the passable width is less than or equalto the preset value, and if the passable width is not less than or equalto the preset value (step S135: N), then obstacle information thatincludes the location of an obstacle but does not include the image istransmitted to the server 200. If the passable width is less than orequal to the preset value (step S135: Y), then as in the firstembodiment, obstacle information in which the location of the obstacleis associated with the image is transmitted to the server 200 (stepS140). In this case, as the preset value, for example, a value obtainedby adding a preset margin to an assumable maximum vehicle width can beadopted. Namely, when it is determined that even a vehicle with thelargest overall width can pass without veering off its lane, thelocation of the obstacle is uploaded to the server 200, but an imageincluding the obstacle is not uploaded to the server 200 (obstacleinformation in this case does not include the image but includes thelocation of the obstacle). As such, in the case of the presentembodiment, for such an obstacle that does not greatly affect passing ofmany vehicles, an image is not transmitted, and thus, an unnecessarilylarge number of images can be prevented from being transmitted to theserver 200.

FIG. 4A is a flowchart of an obstacle information acquisition processperformed by the server 200 of the second embodiment, and FIG. 4B is aflowchart of an obstacle information acquisition process performed bythe in-vehicle system 300 of a guidance vehicle of the secondembodiment. In FIGS. 4A and 4B, processing steps different from thefirst embodiment are shown in bold-line boxes. If the server 200receives obstacle information that includes the location of an obstaclebut does not include an image (determined to be N at step S206 of FIG.4A), then the control part 220 transmits obstacle information includingthe location of the obstacle and an instruction to provideattention-calling guidance to a guidance vehicle located in a presetarea including the location of the obstacle (step S207). In thein-vehicle system 300, if obstacle information received from the server200 includes an instruction to provide attention-calling guidance(determined to be Y at step S306 of FIG. 4B), then the in-vehicle system300 provides attention-calling guidance by providing notification of thelocation of an obstacle before reaching the location of the obstacle(step S325).

On the other hand, if the server 200 receives obstacle information inwhich the location of an obstacle is associated with an image(determined to be Y at step S206 of FIG. 4A), then as in the firstembodiment, the control part 220 performs an image recognition processon the received image, thereby calculating a passable width, andtransmits obstacle information including the location of the obstacleand the passable width to the in-vehicle system 300 (step S210). In thein-vehicle system 300, if obstacle information received from the server200 does not include an instruction to provide attention-callingguidance (determined to be N at step S306 of FIG. 4B), then as in thefirst embodiment, the in-vehicle system 300 performs processes at andafter step S310.

(3) Third Embodiment

In a third embodiment, the control part 120 of the in-vehicle system 100of a probe vehicle further includes a guidance control part 121 d inaddition to the image obtaining part 121 a, the image recognizing part121 b, and the obstacle information acquiring part 121 c. FIG. 5 is aflowchart of an obstacle information acquisition process performed bythe probe vehicle according to the third embodiment. In FIG. 5 ,processing steps different from the first embodiment are shown in boldboxes. In the third embodiment, when the control part 120 of thein-vehicle system 100 recognizes the presence of an obstacle, guidanceof which is to be provided, by performing an image recognition processon a shot image, the control part 120 calculates a passable width basedon the image (step S130). Then, by a function of the guidance controlpart 121 d, the control part 120 determines whether the probe vehicle istraveling in a lane where the obstacle is located (step S145). If it isnot determined at step S145 that the probe vehicle is traveling in theobstacle's lane, then the control part 120 ends the obstacle informationacquisition process. If it is determined at step S145 that the probevehicle is traveling in the obstacle's lane, then by a function of theguidance control part 121 d, the control part 120 determines whether apassing width of the probe vehicle is greater than the passable width(step S150). If the passing width of the probe vehicle is greater thanthe passable width (determined to be Y at step S150), then by a functionof the guidance control part 121 d, the control part 120 provides lanechange guidance to use, as a recommended lane, a lane other than thelane where the obstacle is located (step S155). In addition, if thepassing width of the probe vehicle is less than or equal to the passablewidth (determined to be N at step S150), then by a function of theguidance control part 121 d, the control part 120 providesattention-calling guidance by providing notification of the location ofthe obstacle (step S160). Note that the passing width of the probevehicle has a value obtained by adding a predetermined margin to overallwidth included in the vehicle body information 130 b. According to thepresent embodiment, the probe vehicle having detected an obstacle canprovide guidance for dealing with the obstacle, without through theserver 200.

(4) Other Embodiments

The above-described embodiments are examples for implementing variousaspects of the present disclosure, and various other embodiments canalso be adopted. For example, the obstacle information acquisitionsystem may be a device mounted on a vehicle, etc., or may be a deviceimplemented by a portable terminal, or may be a system implemented by aplurality of devices (e.g., the control part 120 in the in-vehiclesystem 100 and a control part in the camera 140). In addition, thefunctions of the image recognizing part and the obstacle informationacquiring part that are included in the obstacle information acquisitionsystem may be implemented by a server. In addition, the functions of theimage recognizing part and the obstacle information acquiring part maybe implemented by an in-vehicle system of a guidance vehicle. Thein-vehicle system 300 and the in-vehicle system 100 both can also serveas in-vehicle systems of probe vehicles or can also serve as in-vehiclesystems of guidance vehicles. Some components of the above-describedembodiments may be omitted, and the order of processes may be changed oromitted.

The image obtaining part may be configured in any manner as long as theimage obtaining part can obtain a shot image of a road on which a probevehicle is traveling. The camera mounted on the probe vehicle may be acamera whose field of view includes a road ahead of the probe vehicle,or may be a camera whose field of view includes a road behind or a roadon the side. The camera is to be adjusted in advance so that thelocation of an object included in an image shot with the camera (arelative location with respect to the probe vehicle) can be identified.If a relative location with respect to the probe vehicle can beobtained, then an absolute location of an object can be obtained basedon the relative location and an absolute location of the probe vehicle.

The image recognizing part may be configured in any manner as long asthe image recognizing part can detect an obstacle on a road and lanesfrom an image, and may adopt any technique. For example, a determinationas to whether an obstacle is included in a shot image may be made bychecking the image against patterns of obstacles, or may be made usingmodels that are machine trained using images of obstacles as labeleddata. In addition, the image recognizing part may be configured in anymanner as long as the image recognizing part can identify, based on animage, a lane where an obstacle is located among all lanes on a roadsection where the obstacle is located. Note that an obstacle and asection line representing a lane boundary may be detected by using botha camera and a detection device other than the camera (e.g., radar orLiDAR).

The obstacle information acquiring part may be configured in any manneras long as the obstacle information acquiring part can acquire obstacleinformation in which the location of an obstacle is associated with animage, based on a passable width of a lane where the obstacle is locatedin the image. The obstacle may be any thing that can obstruct a vehicleupon passing. The obstacle may be present on a road surface or may be,for example, a dip created in a road. In a case of a dip, when the sizeof the dip is smaller than a reference size or when the depth of the dipis shallower than a reference depth, the dip may be excluded from atarget to be acquired as obstacle information.

The obstacle information may be any information in which at least animage including an obstacle and the location of the obstacle areassociated with each other. The location of the obstacle may becoordinates that directly represent the location of the obstacle or maybe information that indirectly indicates the location of the obstacle.In the latter case, for example, the obstacle information may beinformation in which the location and traveling orientation of the probevehicle at the time of shooting an image including an obstacle areassociated with the shot image.

An in-vehicle system that can communicate with a server may beconfigured to also transmit vehicle location information to the serversuch that the vehicle location information is associated with an ID of avehicle having the in-vehicle system mounted thereon, and may beconfigured to further transmit vehicle body information such that thevehicle body information is associated with the ID of the vehicle. Inthat case, the server can identify a passing width of each vehicle basedon vehicle body information of each vehicle, and can make a comparisonin magnitude between a passable width and a passing width of a guidancevehicle. When a guidance vehicle whose passing width is greater than apassable width is traveling in a lane where an obstacle is located,notification may be provided to an in-vehicle system of the guidancevehicle to provide lane change guidance, and notification may beprovided to an in-vehicle system of a guidance vehicle whose passingwidth is less than or equal to the passable width to provideattention-calling guidance. The in-vehicle systems of the guidancevehicles may be configured to provide guidance according to thenotification.

In addition, although, in the above-described embodiments, obstacleinformation is delivered to guidance vehicles located in a present areaincluding the location of an obstacle, delivery destinations may befurther limited. For example, obstacle information may be transmitted tovehicles traveling on a road section where an obstacle is located, ormay be transmitted, further narrowing down to vehicles traveling in alane where the obstacle is located. In addition, when a route to adestination is already set and includes a road section where an obstacleis located, obstacle information may be transmitted to a correspondingvehicle.

Note that in the above-described embodiments, it is configured such thatthe probe vehicle calculates a passable width by performing an imagerecognition process and the server also calculates a passable width byperforming an image recognition process on an image again, but when apassable width is calculated by the probe vehicle, the server may notperform an image recognition process again. Namely, obstacle informationincluding a passable width obtained as a result of performing an imagerecognition process by the probe vehicle may be transmitted to theserver, and the server may perform a process of the guidance controlpart by adopting the passable width transmitted from the probe vehicle.In addition, for example, a configuration may be adopted in which theserver does not perform an image recognition process on an image, butthe guidance vehicle performs an image recognition process on an imageto obtain a passable width. In this case, obstacle information deliveredfrom the server to the guidance vehicle includes the location of anobstacle and at least one image including the obstacle.

In addition, an image including an obstacle that is received by theserver from the probe vehicle may be used in other processes than animage recognition process for calculating a passable width. For example,the server may perform an image recognition process to identify the typeof an obstacle (a vehicle, a tire, a box, a bag, a dead animal, etc.).In addition, for example, images may be collected as labeled data forcreating, by machine learning, a model for obtaining whether an obstacleis included in an image, a model for obtaining the type of an obstaclefrom an image including the obstacle, etc. In addition, for example, animage including an obstacle may be used to display an obstacle portionof the image in the guidance control part of the guidance vehicle.

In addition, although the above-described embodiments describe anexample in which the number of obstacles included in an image is one,even when a plurality of obstacles are present in the same lane,obstacle information can be acquired based on a passable width. When aplurality of obstacles are present in the same lane, if a distancebetween the obstacles in a direction in which a road extends is lessthan a predetermined value and a distance between the obstacles in awidth direction of the road is, for example, less than a predeterminedvalue, then a passable width may be calculated considering the obstaclesas a single obstacle.

The guidance control part may also provide attention-calling guidance toa second vehicle traveling in a second lane other than a first lanewhere an obstacle is located. Since there is a possibility that a firstvehicle traveling in the first lane where the obstacle is located maysuddenly veer off to a lane (e.g., the second lane) other than the firstlane, attention-calling guidance may be provided. The second lane may bea lane whose traveling direction is the same as that of the first lane,or may be an oncoming lane (no median strip). The second lane may be alane adjacent to the first lane, or the second lane and the first lanemay be next to each other with one or more lanes present therebetween.

Note that not only when an obstacle is present on a road section havinga plurality of lanes, but also when a road section has only a singlelane, guidance for calling attention may be provided by providingnotification of the location of the obstacle. It may be configured suchthat when a passable width at the location of an obstacle is less thanor equal to a passing width of a vehicle on a road section having only asingle lane, the guidance control part provides guidance for avoidingthe road section.

The guidance control part may be implemented by a server. Namely, theguidance control part 221 b of the server identifies a guidance vehicle,selects content of guidance (lane change guidance, calling attention,etc.) to be provided to the guidance vehicle, and determines guidancetiming (a point at which guidance is provided) in the guidance vehicle,and transmits information about the content of guidance and the guidancetiming to the guidance target. The guidance vehicle may be configured tocontrol the user I/F part 344 according to the information to provideguidance.

The passable width may be the length of a portion, in a height directionfrom a road surface, of a lane where an obstacle is located that is notoccupied by the obstacle. For example, FIG. 6 shows an example in whichan obstacle is present on a road with space from a road surface. Such anobstacle may be assumed to be, for example, a branch of a tree, a sign,or a flown object hanging on a branch of a tree or a sign. FIG. 6schematically shows an example in which one of branches of a tree on theside of a road overhangs the road. The image recognizing part detectslanes and an obstacle by performing an image recognition process such asYOLO, pattern matching, or contour detection. In a case of this example,the obstacle is branch and leaf portions overhanging the road. The imagerecognizing part obtains a relative orientation θ and a straight-linedistance D, as viewed from the probe vehicle, with respect to a point P₁obtained by vertically projecting a lower edge portion of the obstacleonto a road surface from above. Then, the image recognizing part obtainsthe location, in the map coordinate system, of the point P₁ of theobstacle, based on the relative orientation θ, the straight-linedistance D, and the current location and traveling orientation of theprobe vehicle. In addition, the obstacle information acquiring partacquires the length of a line segment w_(h) having a lower edge of theobstacle and the point P₁ as its both ends. A correspondence between thelength, in an image, of the line segment w_(h) and the location, in theimage, of a lower end (P₁) of the line segment w_(h) and the length ofthe line segment w_(h) in actual space is identified in advance, andusing the correspondence, the image recognizing part obtains the length(w_(h)) in actual space between the lower edge portion of the obstacleand the road surface. The length (w_(h)) in actual space is the lengthof a portion, in a height direction from the road surface, of a lanewhere the obstacle is located that is not occupied by the obstacle (apassable width in the height direction). The guidance control partallows a guidance vehicle to acquire obstacle information including thelocation of the obstacle and the passable width in the height directionof the lane where the obstacle is located. The guidance vehicle comparesa passing width in the height direction of the guidance vehicle (theoverall height of the guidance vehicle is obtained from vehicle bodyinformation and a height for a preset margin is added to the overallheight, by which a passing width in the height direction is obtained)with the passable width in the height direction, and provides guidanceon the obstacle. For example, when the passing width in the heightdirection of the guidance vehicle is greater than the passable width inthe height direction, lane change guidance is provided to use, as arecommended lane, a lane other than the lane where the obstacle islocated. In addition, for example, when the passing width in the heightdirection of the guidance vehicle is less than or equal to the passablewidth in the height direction, attention-calling guidance that providesnotification of the presence of the obstacle is provided. Note that theobstacle information may, of course, include both a passable width in awidth direction of the lane and a passable width in the heightdirection, and a comparison between a passing width in an overall widthdirection of the guidance vehicle and the passable width in the widthdirection and a comparison between a passing width in the heightdirection of the guidance vehicle and the passable width (w_(h)) in theheight direction may be made, and guidance on an obstacle, content ofwhich is based on results of the comparisons, may be provided.

Furthermore, a technique in which, as in the present disclosure, anobstacle on a road and lanes are detected from a shot image of a road onwhich a vehicle is traveling, and obstacle information in which thelocation of the obstacle is associated with the image is acquired basedon a passable width of a lane where the obstacle is located in the imageis also applicable as a program or a method. In addition, a system, aprogram, and a method such as those described above may be implementedas a single device or may be implemented by using a component sharedwith each part included in a vehicle, and include various modes. Inaddition, changes can be made as appropriate, e.g., a part is softwareand a part is hardware. Furthermore, aspects of the disclosure are alsofeasible as a recording medium for a program that controls the system.Needless to say, the recording medium for the program may be a magneticrecording medium or may be a semiconductor memory, and any recordingmedium to be developed in the future can also be considered exactly inthe same manner.

REFERENCE SIGNS LIST

100: In-vehicle system, 120: Control part, 121: Obstacle informationacquisition program, 121 a: Image obtaining part, 121 b: Imagerecognizing part, 121 c: Obstacle information acquiring part, 121 d:Guidance control part, 130: Recording medium, 130 a: Map information,130 b: Vehicle body information, 130 c: Image information, 140: Camera,141: GNSS receiving part, 142: Vehicle speed sensor, 143: Gyro sensor,144: User I/F part, 145: Communication part, 200: Server, 220: Controlpart, 221: Obstacle information acquisition program, 221 a: Obstacleinformation acquiring part, 221 b: Guidance control part, 230: Recordingmedium, 230 a: Map information, 230 b: Obstacle information, 240:Communication part, 300: In-vehicle system, 320: Control part, 321:Obstacle information acquisition program, 321 a: Guidance control part,330: Recording medium, 330 a: Map information, 330 b: Vehicle bodyinformation, 340: Camera, 341: GNSS receiving part, 342: Vehicle speedsensor, 343: Gyro sensor, 344: User I/F part, 345: Communication part,B: Bounding box, and Bo: Representative coordinates

1. An obstacle information acquisition system comprising: an imageobtaining part that obtains a shot image of a road on which a vehicle istraveling; an image recognizing part that detects an obstacle on a roadand a lane from the image; and an obstacle information acquiring partthat acquires obstacle information in which a location of the obstacleis associated with the image, based on a passable width of a lane wherethe obstacle is located in the image.
 2. The obstacle informationacquisition system according to claim 1, wherein the passable width is awidth of a portion of the lane where the obstacle is located that is notoccupied by the obstacle out of a full width of the lane, and when thepassable width is less than or equal to a preset value, the obstacleinformation is acquired.
 3. The obstacle information acquisition systemaccording to claim 1, wherein when a height of the obstacle is less thanor equal to a reference height, the obstacle information is notacquired.
 4. The obstacle information acquisition system according toclaim 1, comprising a guidance control part that allows a guidance partto provide guidance, based on the obstacle information, wherein theguidance control part allows a guidance vehicle that travels in the lanewhere the obstacle is located and that has not yet passed through thelocation of the obstacle to: acquire the obstacle information includingthe location of the obstacle on a road and information indicating thepassable width of the lane where the obstacle is located; and provideguidance on the obstacle, based on the location of the obstacle and thepassable width.
 5. The obstacle information acquisition system accordingto claim 1, comprising a guidance control part that allows a guidancepart to provide guidance, based on the obstacle information, wherein theguidance control part allows a guidance vehicle that travels in the lanewhere the obstacle is located and that has not yet passed through thelocation of the obstacle to: provide notification of presence of theobstacle; and provide lane change guidance to use, as a recommendedlane, a lane other than the lane where the obstacle is located, when apassing width of the guidance vehicle is greater than the passablewidth.